Add `copy.replace` for 3.13 by max-muoto · Pull Request #12262 · python/typeshed

max-muoto · 2024-07-03T00:27:04Z

Add copy.replace which is new in 3.13: https://docs.python.org/3.13/library/copy.html#copy.replace

max-muoto · 2024-07-03T00:29:50Z

stdlib/copy.pyi


+if sys.version_info >= (3, 13):
+    def replace(obj: _T, /, **changes: Any) -> _T: ...
+    __all__ += ["replace"]


See us mostly using += in most places with __all__, guessing this is for better static analysis support, or something like that?

Yes, this is fine, it's so we don't have to repeat the whole list for every version.

Oh, I meant versus using append.

Except it actually isn't in __all__: https://github.com/python/cpython/blob/89bf33732fd53fbb954aa088cfb34f13264746ad/Lib/copy.py#L59 . So we shouldn't add it in the stub either.

This may be an oversight, so consider opening an issue over on CPython for adding replace to copy.__all__. But as long as it's not there at runtime, we shouldn't add it in the stub either.

My bad, let me fix that.

max-muoto · 2024-07-03T00:43:55Z

stdlib/copy.pyi

 def copy(x: _T) -> _T: ...

+if sys.version_info >= (3, 13):
+    def replace(obj: _SR, /, **kwargs: Any) -> _SR: ...


Higher-kinded types could be cool here, but obviously no way to get great kwargs type-checking. Even in that case you can't use P.kwargs without P.args.

max-muoto · 2024-07-03T00:44:09Z

Some classes are missing __replace__ dunders but will tackle that separately.

max-muoto · 2024-07-03T01:35:19Z

Interestingly, Pyright is fine with this example, and MpPy is not (causing the test failure):

from __future__ import annotations
from typing import Self, Protocol, ClassVar
import copy


class ReplaceableClass:
    def __init__(self, val: int) -> None:
        self.val = val

    def __replace__(self, val: int) -> Self:
        cpy = copy.copy(self)
        cpy.val = val
        return cpy


from collections.abc import Callable


class _SupportsReplace(Protocol):
    __replace__: Callable[..., Self]



def foo(sr: _SupportsReplace) -> None:
    ...

foo(ReplaceableClass(23))

Would you consider this a MyPy bug @JelleZijlstra?

max-muoto · 2024-07-03T01:46:08Z

Interestingly, Pyright is fine with this example, and MpPy is not (causing the test failure):

from __future__ import annotations
from typing import Self, Protocol, ClassVar
import copy


class ReplaceableClass:
    def __init__(self, val: int) -> None:
        self.val = val

    def __replace__(self, val: int) -> Self:
        cpy = copy.copy(self)
        cpy.val = val
        return cpy


from collections.abc import Callable


class _SupportsReplace(Protocol):
    __replace__: Callable[..., Self]



def foo(sr: _SupportsReplace) -> None:
    ...

foo(ReplaceableClass(23))

Would you consider this a MyPy bug @JelleZijlstra?

Using paramspec is a workaround that works for both: 58bb691

erictraut · 2024-07-03T04:50:42Z

I think there's a problem with the way you defined the _SupportsReplace protocol. You're defining __replace__ as an instance variable.

I recommend changing it to:

class _SupportsReplace(Protocol):
    def __replace__(self, *args: Any, **kwargs: Any) -> Self:
        ...

Mypy works fine if you define it this way.

max-muoto · 2024-07-03T04:52:47Z

I think there's a problem with the way you defined the _SupportsReplace protocol. You're defining __replace__ as an instance variable.

I recommend changing it to:
class _SupportsReplace(Protocol):
    def __replace__(self, *args: Any, **kwargs: Any) -> Self:
        ...
Mypy works fine if you define it this way.

Thanks, I think I thought originally this wouldn't work since it doesn't work if you leave out *args with Pyright.

Fixed: 860f674

max-muoto · 2024-07-03T17:38:25Z

Added replace to __all__ based on python/cpython@7c66906

JelleZijlstra · 2024-07-03T17:40:55Z

CI will fail for now since it's not in __all__ in the last beta. We can either wait for the next beta or merge this without __all__ for now (which would be my preference).

picnixz · 2024-07-04T13:29:35Z

stdlib/copy.pyi

+_SR = TypeVar("_SR", bound=_SupportsReplace)
+
+class _SupportsReplace(Protocol):
+    # In reality doesn't support args, but there's no other great way to express this.


Technically speaking, it does support positionals, but those will be passed as keywords. What it does not support however is positional-only arguments.

On the other hand, you can just force people to use keyword-only arguments so that it matches https://docs.python.org/3.13/library/copy.html#object.__replace__. Or at least, make self a positional-only argument.

On the other hand, you can just force people to use keyword-only arguments so that it matches

The only issue is that won't work that way you might think with either MyPy or Pyright. Take this example: https://pyright-play.net/?pythonVersion=3.13&strict=true&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAK4MYAxmADYA0UAginAFDOkUCGAzh1APoDKAVwQ48HAEoBTBO1ISAFETAlyFAJQAuRlG1QAJhOC8eIKTIk8ecjhIrAaAKnsBrAO5sQaDuroNVUALQAfFAAcmAoEt4AdDHMjKyc3ACS2BSaOnoGRibSbLIWVjZ2UO5oPACM3qgwfkGh4ZFQMVHM%2BoYwEhwwcjkybABGFI38QiIw4qZ5ErXBYRHRscxY0lAAvFAp0nKqjB1dcstqQA

Yes, callables with *args: Any, **kwargs: Any are treated specially in the spec.

picnixz · 2024-07-04T13:35:23Z

Some classes are missing __replace__ dunders but will tackle that separately.

We just added __replace__ on AST nodes (it's not in 3.13) in case you're interested in adding it for 3.14 only.

max-muoto · 2024-07-04T16:16:52Z

Some classes are missing __replace__ dunders but will tackle that separately.

We just added __replace__ on AST nodes (it's not in 3.13) in case you're interested in adding it for 3.14 only.

My guess is we'll get around this once we start running stubtest on 3.14.

JelleZijlstra · 2024-07-12T00:14:11Z

stdlib/copy.pyi

+_SR = TypeVar("_SR", bound=_SupportsReplace)
+
+class _SupportsReplace(Protocol):
+    # In reality doesn't support args, but there's no other great way to express this.


Yes, callables with *args: Any, **kwargs: Any are treated specially in the spec.

max-muoto · 2024-07-23T01:12:06Z

@JelleZijlstra It looks like we're running against beta 4 now, do we want to merge this to resolve the CI failure?

github-actions · 2024-07-23T01:25:36Z

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

JelleZijlstra · 2024-07-23T01:34:27Z

Sure. Seems like there's some other unrelated new stubtest failures too.

max-muoto commented Jul 3, 2024

View reviewed changes

max-muoto marked this pull request as draft July 3, 2024 00:30

This comment has been minimized.

Sign in to view

max-muoto commented Jul 3, 2024

View reviewed changes

max-muoto marked this pull request as ready for review July 3, 2024 00:44

This comment has been minimized.

Sign in to view

max-muoto mentioned this pull request Jul 3, 2024

__replace__ should be synthesized for dataclasses in 3.13 microsoft/pyright#8300

Closed

This comment has been minimized.

Sign in to view

max-muoto mentioned this pull request Jul 3, 2024

Dataclasses should define __replace__ in 3.13 python/mypy#17471

Closed

This comment has been minimized.

Sign in to view

picnixz reviewed Jul 4, 2024

View reviewed changes

max-muoto mentioned this pull request Jul 6, 2024

namedtuple.__replace__ should return Self python/mypy#17497

Closed

JelleZijlstra approved these changes Jul 12, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

max-muoto added 3 commits July 22, 2024 20:10

Add copy.replace

892c7b8

Add to __all__

b13a641

Tweak

c0bc3f9

max-muoto and others added 19 commits July 22, 2024 20:10

Append to all

f0a22fa

Use +=

ef0758e

Fix

b6ed3d5

Tweak

731e8b2

Add test

6cf408c

[pre-commit.ci] auto fixes from pre-commit.com hooks

5a82666

Use Self from typing extensions

58528bb

Use TE

19a1f9f

[pre-commit.ci] auto fixes from pre-commit.com hooks

294c7c0

Remove from __all__

94901c0

Fix

d4b52ee

changes > kwargs

bdf3f1d

Fix mypy support

dc40a89

Remove unused

59eb56c

Fix

a19f672

Address Eric's feedback

c2fd7ec

Add replace to all

5348837

Add replace

23f4d66

revert

b404c60

max-muoto force-pushed the add-copy-replace branch from 99de1e7 to b404c60 Compare July 23, 2024 01:10

Add for all

c3fd56a

This comment has been minimized.

Sign in to view

redundant self

9c92081

JelleZijlstra merged commit 2909053 into python:main Jul 23, 2024

max-muoto mentioned this pull request Jul 23, 2024

Remove ParamSpec for SupportsReplace #12410

Merged

max-muoto added a commit to max-muoto/typeshed that referenced this pull request Sep 8, 2024

Add copy.replace for 3.13 (python#12262)

d0f65a6

Uh oh!

Conversation

max-muoto commented Jul 3, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

max-muoto commented Jul 3, 2024

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

max-muoto commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

max-muoto commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

erictraut commented Jul 3, 2024

Uh oh!

max-muoto commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

max-muoto commented Jul 3, 2024

Uh oh!

JelleZijlstra commented Jul 3, 2024

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

max-muoto commented Jul 4, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

max-muoto commented Jul 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

github-actions bot commented Jul 23, 2024

Uh oh!

JelleZijlstra commented Jul 23, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

max-muoto commented Jul 3, 2024 •

edited

Loading

max-muoto commented Jul 3, 2024 •

edited

Loading

max-muoto commented Jul 3, 2024 •

edited

Loading

picnixz commented Jul 4, 2024 •

edited

Loading

max-muoto commented Jul 23, 2024 •

edited

Loading